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ABSTRA ^ 

An external evaluatict was conducted of the 
Picturepaqes television work-alonq program in Kentucky. Tfce program 
consisted of 100 televised lessons and bccklets, an^ was given to 237 
preschool children. The lessons presented "games" mhich were designed 
to develop coqni ti ve concepts and skills. Eoth process and outcome 
evaluations were made. Process evaluations assessed vbetber televised 
portions and booklet distributions were implemented according to 
plan. Booklets were distributed as intended tut several factors 
affected availability of the televised component. Outcome evaluations 
focused oijf the program •s effect on children's learning readiness, 
attitudes, and skills, and on parent-child interactions in hcme-based 
and preschool enter- based studies. The results of the home-based 
study indicated that -parents felt children learned from the programs, 
but the center bas^d study offered no support for program effects cn 
children's learninq readiness skills. Constraints on the program were 
discussed and several recommendations were offered- <Author/BD) 
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Evaluation of a-Televised Work-Along 

Program for Preschoolers ( 

This gap^r *>eports on the third-party evaluation of a , statewide im- 
plementatiop of the Picturepages p^dfgram in KefhtUcky. This innovative 
preschool prtigram presented through public media Was implemented by the 
Kentucky Association of Community Action Ag^^cies (KACAA) with the^^on- 
sorship of -the Community Se/v i ces Agency (CSA) , Region IV. Research for 
Better Schools (RBS) . des i gned and conducted art independent ev^luatiqm. \^ 
The challenges in conducting such *an evaluation were many,, given t^e 
statewide public dissemination of the program, the preschool audience, 
and the relatively brief treatment period {20 weeks). An extensivei* report 
on this«evaluat ion has also been prepared by R&S (Ousewicz 6 Coller, 

1978).-^ A " ^ : 

The P i ct urepages^Cbncept 



The PictArepages series was conceptualized and designed to meet the 
critical need f^r early childhood education in the cognitive domain, a 
need which has befen ^ i dent i f i ed and wi^dely documente^^cf . Pi ct^epages was 
developed and copyrighted by C^mpulMrn, Inc. of Pittsburgh to address^ 
these needs by providing a structuM learning experience for children, 



This paper is a condensed version of a much larger two-vo/ume re-* 
porlgpcopies of which will be made ayaiJable through the ERIC system: 
An Evaluation of the Kentucky Picturepag es Program (Dusewicz 6 Colleri 
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■ ut i 1 i z i ng a pres^en tat ion mode (te 1 evi s ion) that makes ^.h^program ^va^ 1 - 
able to rDos t chi Idren . In add i t i on , mul t i -col o red i 1 1 ust rated newspr i nt 
booklets are provided to each child so that they may work along with the' 

... - • n\ 

act I V I 1 1 es^presented on TV. 

Picturepages is usually presented as a segment of a children's TV 

program .vith the local personality showing the children, step-by-step, 

how to use their P i c t ure^i^ges for that day. Though .col orful and fun for 

children to use, Picturepages is conceived as a very real t each i ng too 1 ' 

that involves the child in specific learning activities. While the TV 

host makes the dally Picturepages lesson presentation, the preschool 

child, alone or with his/her parent, actively follows along*at home by 

usi>cig a Picturepages booklet which is distributed in advance of the 

broadcast through public distribution points. Each booklet contains five 

daily lessons which coincide with the broadcasts for that week. Each ' 

booklet also contains a section explaining the lessons and a section 

listing activities to be used by parents or others to supplement the ^ 

concepts anS skills presented by the TV host. This^ latter feature enables 

the child and parents, or teachers, to continue playing the "game" to / 

develop further the concepts and skills presented in, the. /TV Program. 

The Picturepaqes concept w^s developed with an understanding of the 

''^ ' ~ . 

importance of early development of'cognitive skills as the basic tools of 

learning. It recogn.izes the need .to utilize the env i ronmen t o^ th^ child 

in developing skills: home, pa n^ent , and peers. It recognizes tbe need . 

to, react;) large numbers of preschool chfldren and the ef f i cacy of \is i ng 



tfae medium of television as the vehicle for accomplishing this end. It 
acknowledges the specific need to reach lower income groijps and disadvan- 
taged child ren • ^ 

The f i rst Picturepages series went on the air commercially in 1973. 
This w^'s fallowed a year later by the first of a series of Community 
Action Agency sponsored programs using the Picturepages concept. Now, 
within the span of just a few years, a total of six urban cities 
(Pittsburgh, PA; Columbia, SC; Memphis, TN ; Birmingham, AL; Miami, PL; 
Little Rock, AK) and the entire Commonwealth of Kentucky have implemented 
the Picturepages Program. - • 

The Kentucky Implementation of Pictu repages' 

In adapting the Picturepages concept for use in Kentucky, prepara- 
tion of^^a specially selected set of materials and broac^'casts was necessary 

as well as the institution of a number of 6rgan i za t i ona )^echan i sms for 

if 

4' 

implementing the program. While some of these mechan i srr)^ • were similar to 
those used in prev ious/ P i c t urepages implementations, ot^^rs represented 
new approaches to handling the large scale program operations required. 

The Kentucky program represented the first state-wide adaptaticin of 
Picturepages. No less than 12 independent public and private organiza- 
tions and groups of organizations wej^ involved in the implementation oV 
the Kentucky Picturepages program. yThe Community Servi.ces Administration 
(formerly OEO) fi^ded a rpajority nf the program. Compulearn, the orig- 
inator of Picturepages, developed, produced, and managed primary delivery 
of 20 weeksi of a specially selected set of Picturepages booftfets. 



Ifi Kentucky, delivery of the booklets to distribution centers was 

* 

handled by a private trucking firm. The Kentucky Association for Community 
Action Agencies (KACAA) coordinated the program state-wide, created 
awareness for it, anc ^-rved as a public information clearinghouse. 
Twenty-one (21) regional community action agenc i es --gene ra 1 1 y identified 
with sponsorship of community programs i^n nutrition, home improvement, 
child care, job tra'ining, Headstart , social and health services--served 
as distribution sites fo/ Picturepages booklets. Other distribution 
agencies included Long John Silver's Seafood Shoppes , publ/c sites (e.g., 
schools, libraries, and Department of Human 'Resources ) and individual 
commercial, professional, and other pr i vate ..e^tab 1 i shment s (e.g., stores, 
doctors' offices, and churches, etc.). ^ 

The Abbott Advertising Ager^cy, national advertising headquarters for 
Long John Silver's Seafood Shoppes, served a# liaison between the shoppes 
serving as distribution points and the ot h^r, part i c i pat i ng agencies. The 
Abbott Agency also developed and a rranged Sponsored cotrnie rc i a 1 s \i ? ch 
prorn^ted the program. Seventy-five (75) indfvF^al shoppes, Jocated 
throughout Kentucky, served as d i st r i but i oi\ s i t«4 for the Picturepages 



\ 

booklets. L 



Happy and Froggie, two TV personalities, provi^d televised lessons 
far each of 100 picturepages. WTVQ of Lexi/igton; over whose station the 
Happy and Froggie Show is normally broadcast, videotaped the 100 five- 
minute lessons and supplied the tapes to other stat ion s to be involved 
^in the program. The televised lessons themselves w^re^to be broadcast 
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overr both the public KET Network, consisting of ^ s^tions and trans- 
lators, and private stations: V/TVQ in Lexington, WDRB i n \ou i sv i I I e and 
WKYH in Hazard. 

Research for Better Schools was under cont r^^i^^^h 4^AA to conduct 
an evaluation of the Kentucky Picturepages ii()pf^m^r*tat ion . The evalua- 
.tion was to address implementation proceS^s qUrstions and program impact 
questions. • ' ^ .7' 

Methodology 

■ v^Thef eval uat ion plan called A^r bot\ /process and outcome ^valuation' 
components. The 6utconie com6onelS|t was further divided into a home-based 

^ — 

Study and a center-based study^.j T)\dr procedd^es employed in each component 
are briefly described, ^he primary focus was upon the question of program 
effectiveness and secorudarily upon program implementation as a context 
for interpreting outcoml^s. . "^^^^ ' — 

Proems Eval uation . Th i component was intended to answer the fol- 
lowif<y questions: \ 

• Is the tekevised portion of the (program imple- ' 
mentedaccordingtoplan? 

• Is the ma«rials portion of the program imple- 
mented according to plan? - 

_ Over the course of program impl ejTien ta t ion , attempts were made to 

^^^^sess the progress of materials dissemination including numbers of book- 

lets received and distributed and reports of difficulties from each dis- 

\ 

se||_{nation ptoint in getting booklets ^o the target population. Attempts 
were also made to assess tKe^extent of proper implementation of the 
television broadcasts over the program period. 



These imp 1 emenlfat ion monitoring activities were conducted by pro- 
viding all distribution points with forms for reporting the flow of book- 
lets. A sample of distribution centers was also contacted by telephone 
to review their experiences. The TV broadcasts were rnonitored primarily 
by the nu rsery school s involved in the center-based outcome evaluation 
component . 

Outcome Eval uat ion . The outcome evaluation was primarily intended 
to answer the following questions: 

• Does the program affect children's learning readi- 
ness at t i tudes? 

• Does the program affect 1 ea rn i ng -re 1 ated parent- 
childinteractions? ^ 

• Does the program affect children's learning readi- 
ness skills? 

Two evaluation approaches were utilized in assessing program effec- 
tiveness: A Home-Based Study and a Center-Based Study. 

The Home-Based Study was conducted to determine program effects on 
children's learning readiness attitudes and on learning-related parent- 
child interactions. This study involved parent responses to a single- 
l^^ge survey questionnaire. A total of 50,000 questionnaires were dis- 
tributed throughout Kentucky. 

The Center-Based Study, employing a t rue -expe r i men ta 1 design, was 
focused upon determining program effects upon children's learning readi- 
ness skills. This study in^lved a todal of 237 participating children 
aged two to five randomly assigned to experimental and control subject 



groups. These subjects were drawn from several child care center sites 
within, the state. All participants were pre- and posttested on a variety 
of carefully selected instruments. 

In every evaluation study the selection of instruments to assess the 
effects of the experimental treatment is of central importance-- i n this 
study Jt was thought to be crucial. Not only was the Kentucky Picture- 
pages Program to be of 'rather low intensity (8 1/3 program hours distrib- 
uted- over 20 weeks and many subject areas) but the group to he used as a 
control group was to receive an educational treatment as well. fy was 
anticipated that if any differences were to occur, they would be small 
and closely tied to the content of the program itself. The Jnst rument 
selection was then conducted in accordance with a we 1 1 -deve loped selection 
mode I . ^ . ' 

The following norm-referenced measures were employed: the Peabody 
Picture Vocabulary Test, the Cooperative Preschool Inventory, and the 
Walker Readiness Test for Disadvantaged Preschool Children. Three domain- 
referenced measures were also selected from the Early Childhood Inventory 
collection: the Prepositions Inventory, the Relational Concepts Inventory, 
and the Shape Name Inventory. 

Resu 1 ts 

Ihe process and outcome results are presented separately below. 

Process Evaluation . The results related to booklet distribution were 
limited due to incomplete data. Many of the d i s t r i bu t^Ion centers were 
unable to maintain the prescribed records. Several distribution problems 



( 

were noted and referred to KAGAA for remediation. It appeared that most 
ofTt^he booJ<lets were distributed as intended. 

The televised portion of the program was implemented statewide for 
the first 10 weeks and on a limited basis for the second 10 week period''^ 

9 

After 10 weeks, the KET Network of 13 stations and seven translatQ'rs de- 
cided to discontinue the Picturepages broadcasts. Except for cable TV 
accessibility, this left only three remaining broadcast stations located 
in Louisvili^e, Lexington, and Hazard. Further, the Louisville station 
^ changed the broadcast time by five minutes and the Lexington station 

dropped its morning broadcast of Picturepages. These changes seriously 
affected the availability of the televised component of the program. 

Prompted no doubt by the consequent lack of broadcasts to several 
outlying areas in Kentucky, there was a simultaneous decision by the Long 
John Silver's Seafood Shoppes to discontinue distribution of the picture- 
pages booklets in areas other than the three indicated above. This left 
approxiniatel y half of the original shoppes participating during the second 
10 weeks of the program. The second half of the implementation was thus 
a subsw<tant ia 1 ly reduced version of the original plan. 

r 

Outcome Evaluation^ The Home-Based Study yielded a total return of 
1,067 questionnaires, or 2.13% of the 50,000 survey forms sent to primary 
distribution centers throughout the state. Results of single-ftem and 
inter-item tabulations produced the following principal findings: 

-8- 



• Children in the age rarjpbs of two through seven 
(or older) pa rt i c i pate^i n the program with the 
highest proportion of participants being four and 
f i ve yea rs of age . 

• A large proportion of the respondents not only 
engaged in the follow-up activities described in 

the Picturepages booklets but also indicated that % 
they actually teach their children nxjre as a r'e- 
suit of their involvement in the Picturepages 
program." 

• Parents felt that 'it was easier to work wi^h and 
\ teach their children as a result of their experi- 
ence with Picturepages and this perception was 
highly associated with the number of weeks they 
engaged in follow-up activities. ' 

• Children working with Picturepages at home seem to 
their parents to enjoy all components of the pro- 
gram: the booklets, the TV broadcasts and the 
follow-up activities. ^ 

• Children working with Picturepages at home seem to 
their parents to be learning from their involvement 
with the Picturepages program. . 

• A large proportion of the responder^ts indicated that 
they would want Picturepages to continue; fewer in- 
dicated that they would be willing to pay for the 
prog ram . 

In the Center-Based Study, after 20 weeks, 110 of the original 237 
participating children had completed pre- and posttest data sets. Based 
on the test measures administered, a total of 1^ variables were identified 
for analysis. Repeated measures analyses of variance wer^ conducted on 
all variables utilizing the treatment by repeated measures interaction 
effect as the indication of treatment impact. Result^of these analyses 
indicated no evidence of significant program effects with respect to any 
of tha variables. If indeed the Picturepages program enhances children's 
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lear-nincj readiness skills, it was not evident in data obtained from the 
Center-Based Study . 



Di scuss ion 
- — r- 



A major question addressed by this study was whether or not the 
Picturepages Program affects children's 1 ea rn i ng„. read i nes s skills. Re- 
sults from both the home-based and center-based studies have pertinent 
i mp 1 i cat i ons . 

in the home-based study, tabulations of responses on the Parent Sur- 
vey indicated that nearly 80 percent of the ch i 1 d ren 'pa r t i c i pa t i ng in the 
Picturepages program seemed to their parents to have learned from their 
involvement. While this may appear on the surface to be a considerable 
erxiorsement of the program's effectiveness, it is, in fact, difficult to 
place any great confidence in this finding. This is due to the lack of 
experimental control over the conditions under which the home-based study 
was conducted. Thete are at least two alternate arid equally viable in- 
terpretations of the findings: (1) what parents may have been observing 
was a general ma tu rat i ona 1 or growth effect in their children unrelated 
to program participation; (2) a "halo" effect produced by generalized 
enjoyment of the program and positive responses on other items may have 
oe^n responsible for carrying over a positive response set to this item. 

The center-baspd study offers no support at all for any program- in- . 
duced effect upon children's learning readiness skills. Extensive analy- 
ses of pre- and posttest data on all relevant variables indicated no 
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statistically significant program vs. control differences which could be 
ascribed to the Picturepages experience. \./hile the results from the home- 
oased study should not necessarily be interpreted as supportive of the 
Picturepages prograr^, results from the center-based study should not 
necessarily be interpreted as con t ra - SL^I^rt i ve of the program. There 
.-.ere seve^il t hodo I Cr , I c.i 1 constraints upon conduct of the center-based 
tud> .vhich undoubteJly mitigated against program effectiveness. These 

includea the following: ^ the ill-timing of program implementation to occur 

/ 

over the summer months wh'ch are typically characterized by high turnovers 
ana vacations at child care centers; the po.irly scheduled broadcast time 
for tne TV segment insofar as viewing at the child care centers was con- 
cerned: and the adf- i n i s t ra t i ve and staffing problems associated with con- 
ducting a controlled study in child care centers, involving the random 
assignment a-.^ segregation of children into experimental and control 
grour and malnta-ni-^g the integrity of those groups. 

'^r^' additional constraint was the program itself. The Kentucky Pic- 
:^re::aqes Pr;.:jrar was des"gnec3 as a 20 weeK sequence consisting of 100 
vc -ip^te televised work-along lessons. Discounting any of the follow- 
ac:'vtties, the prograr being studied was only 8 1/3 hours in duration 

t-.e lessons Deing distributed over a 20 week period. Without ques- 
t ( or , :he KentucKv progra^ was one of low intensity. Therefore, interpre- 
tations Dased on the results of this study may on4y be appropriate for 
low intensity programs of short duration; they do not necessarily allow 
one to generalize to, for example, a higher intensity, forty-week program 
involving the same content and approach. 
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Finally, it should be emphasized that part of this study took place 
in^child care centers, wh^rt* all children (experimental and control) 
received an educational treatment. This treatment probably produced edu- 
cational effects which made the incremental effects of Picturepages harde 
to detect. Tnis factor also means that results may not be genera 1 i zabl e 
to effects which would have occurred in a home environment. 

Based on t^is ev^aluation experience with the Kentucky implementation 
Picturepages several recommendations were offered: 

• The Pictu re pages concept 'appears to have sufficient 
educational potential that its effectiveness should be 
stud ied further . 

• Future evaluations of Picturepages should focus On 
determining effects under more intensive treatment 
cond i t i on s . 

• A content analysis should be performed on each se- 
lected Picturepages sequence prior to its finaliza- 
t - on in order to assure non-bias and uniformity of 
content emphasis from one program implementation to * 
anot he r . 



• Prior to each implementation, field-testing of both 
broadcasts and book lets should be conducted on small 
samples for their entertai nment and effect iveness 
value and to eliminate problems of production quality. 

• Contracts, assurances or guarantees should be secured 
from both TV broadcast stations and booklet distribu- 
tion centers for participation over the full duration 
of the planned Pictu re pages p rog ram and for full coop- 
eration in providing feedback on the broadcast and 
distribution process as well as cooperation in any 
evaluation efforts. 
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